Experimenting with Anomaly Detection by Mining Large-scale Information Networks

Social networks have formed the basis of many studies into large networks analysis. Whilst much is already known regarding efficient algorithms for large networks analysis, data mining, knowledge diffusion, anomaly detection, viral marketing, to mention. More recent research is focussing on new classes of efficient approximate algorithms that can scale to billion nodes and edges. To this end, this paper presents an extension of an algorithm developed originally to analyse large scale-free autonomic networks called the Global Observer Model. In this paper, the algorithm is studied in the context of monitoring large-scale information networks. Hence, taking into account the size of such networks, the proposed algorithm starts by partitioning the graph using structural network metrics. This is followed by a calculation of the graph nodes’ metrics, which are used in the selection from the original graph a subset of nodes to be monitored. The paper is organised as follows: it will outline the problem definition and algorithm, then will proceed to a brief description of an event and signature based model used to instrument monitored nodes. Finally, the paper will conclude with an evaluation using an infection detection scenario, which will be followed by a general discussion and proposed further work.